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AESTRACT 

The learning effect of instructional objectives 
treatment in economics was tested by seeing if significant 
differences existed in mean achievement scores of the economic 
understanding of students using the treatment and those with no 
treatment. During the 1970 fall semester, 300 students from three 
community colleges in California (Cypress, Mount San Antonio, and 
Orange Coast) participated in this study. The evaluative instrument 
used was the "Test of Economic Understanding, Form B. " Information 
was provided concerning student characteristics and comparisons of: 
post- test differences, pre-tests for group differences, pre-test to 
post- test gain differences, and pre-tests for withdrawal effects. The 
overall conclusion is that instructional objectives do produce 
definite learning effects, though the level of significance may be 
less than sufficient to justify efforts and resources used for such 
teaching methodology. Additional research is needed to further test 
the effects of instructional objectives. A copy of the instructional 
objectives is available, upon request, from the author. Cypress 
College, California. (CA) 
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Introduction to the Study INFORMATION 

What is "economic understanding," ^ and how do we know when a student has 
attained a "satisfactory level" of such learning? Further, what teaching 
methodologies are most efficient in attaining this goal for our students? Do 
we, as teachers of economics, define clearly for our students what concepts 
they must master to achieve economic understanding, and do we know what sub- 
concepts are essential in mastering different levels of learning? 2 

All of these questions must surely confront and confound any teaching 
economist facing his students, and were in fact the very questions spurring 
this research study. For in the literature of education high pi'aise is often 
paid the use of "instructional objectives" in achieving learning goals by such 
authorities as Glaser, 3 Kapfer, ^ Krathwohl, 5 GagnC, ^ and Mager. 7 Learning 
is to be much improved if learning goals are clearly stated to the student. 
Specifically, such statement should include (a) the situation in which the 
learning behavior is to occur, (b) a precise description of the behavior 
expected, (c) the object or goal of behabior, and (d) a specified criterion 
performance level. Indeed, high claims are made for learning improvement via 
the use of instructional objectives, regardless of instructional area. 

It would perhaps be expected, given such claims, that a survey of the 
literature (both in education and economic education) would thus turn up 
many research studies on both the design and learning effect of instructional 
objectives. ® This expectation, however, is far off the mark. For few 
research studies have been completed to test the learning effect of instructional 
objectives. ^ This dearth of research reporting is summed up pointedly by 



Eisner when he states that: 
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What of the research on educational objectives?. . .A 
number of questions can be asked about educational 
objectives that are in principle amenable to empir- 
ical study. . . . 

When one looks for research on these questions, one 
soon finds that for the most part they havo been 
neglected. . .In view of the admonitions in curriculum 
literature to state objectives in behavioral terms, 
it is surprising to find such a paucity of empirical 
studies available. 

Such research deficiency would be of little concern were it not that 
improvement of economic understanding may be viewed as strongly dependent 
upon such research. The importance of such instructional objectives is 
noted by Joseph in summary of a meeting designed to improve the teaching 
of economics: H 

After the stimulating discussion at the Merrill Center, 

I am firmly convinced that there is a strong need for 
a critical evaluation of the objectives of the basic 
economics course. Without an explicit goal to use as 
a standard, these courses can range from pallid discus- 
sions of current events to non-functional exercises in 
applied mathematics. 

To further add emphasis to the need for instructional objectives research, 

especially as related to economics instruction at the community college level, 

the Thompson/Walthall/Merson Report clearly calls attention to such deficiency: 

A review of published objectives in economics education 
concludes that accepted definitive statements as to the 
specific educational intents of economics instruction 
are lacking and that little can be accomplished in 
improving economics education until spedific behavioral 
objectives attuned to the junior college setting can 
be formulated. 

Economists often disagree on instructional objectives. 

However, it is axiomatic that we cannot select course 
content or evaluate the effectiveness of differing 
instructional practices until agreement is reached on 
the level of achievement we expect students to attain. 

Preparation of acceptable lists of behavioral objectives 
stated in measureable terms must proceed any experimenta- 
tion with the instructional process. 



The Research Study 



Given the proceeding introductory comments, the problem to be researched 
was that the learning effect of instructional objectives treatment was undeter- 
mined. In order to research such effect, two null hypotheses were set up: 

1* Ho^: There will be no significant difference (Z=.05; 1-tail) 

in mean achievement scores of economic understanding at end 
of course instruction between selected community college 
students receiving treatment of instructional objectives 
(experimental groups) and those receiving no treatment 
(control groups). 

2. Ho£: There will be no significant difference (Z-.05; 1-tail) 

in changes of mean achievement scores of economic understand- 
ing from beginning to end of course instruction between 
selected community college students receiving treatment of 
instructional objectives (experimental groups) and those 
receiving no treatment (control groups). 

By testing such hypotheses, the effect of instructional objectives on learning 

outcomes would be measured in terras of final scores (Ho^) and gains in scores 

from pre-test to post-test means (H 02 ). 

Perhaps the most important step in conducting the research was the develop- 
ment of a list of instructional objectives. The goal was to construct a list 
which was specific enough to elicit learning gains from the student, yet be 
general enough to be acceptable to a range of participating colleges and 
economists. Such a list was constructed for the one-semester "survey" course 
of economics principles via reference to the Task Force Report,^ the 
Thompson/Walthall/taerson Report, ^ a wide variety of Joint Council Onr 
Economic Education literature, and a Participating Group of community college 
economists at the Center For Economic Education in California State College, 
Fullerton* (This list is available upon request from the author.) 

In conjunction with development of the instructional objectives list, the 
Test Of Economic Understanding (TEU - Form B) ^ was chosen as the evaluative 
instrument* The TEU appeared most appropriate for a one-semester "survey" 
economics course, was easily administered and accepted by participating 
economists, and was nationally-normed for comparative purposes* ^ The 



research design incorporated the TEU on botn a pre-test and poet-test basis, 
so as to test the null hypotheses previously described. 

The research design, and treatment of instructional objectives itself, 
involved three community colleges in Southern California (Cypress, Mount San 
Antonio, Orange Coast) with some 300 economics students, and was conducted 
during the Fall semester of 1970. In each college a control group was taught 
by traditional methodology, while the experimental group received copies of 
the instructional objectives and were periodically tested (programmed basis) 
as to their performance at criterion level. Both groups were taught with 
similar texts, meeting times, rooms, etc. by the same instructor so as to 
reduce "contributory factors" to learning results other than instructional 
objectives. And both groups in all colleges received a pre-test and post-test, 
using the TEU instrument (See Table II). 

Test Findings 
and Conclusions 

As an overall summary of test results, one can say that instructional 
objectives do produce definite learning effects, though the level of significance 
may be less than that deemed sufficient to justify efforts and resources to such 
teaching methodology. And an overall conclusion would be that additional research 
is needed to test instructional objectives effect, especially with research design 
regard to possible "impurities" encountered in this study and described below. 

To show the effect of instructional objectives on end-of-semester learning, 
Table I compares post-test results of control and experimental groups. While 
results vary among colleges, the overall difference of a Z-sccre of +1.54 gives 
a .06 level of significance to instructional objectives treatment* Such a result 
is perhaps "fair" for new instructional methodologies, but was not sufficient to 
reject Ho^ at the required .05 level of significance* Therefore, this research 
study would point to definite learning improvement from instructional objectives 
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TABLE I 



COMPARISON OF POST-TEST DIFFERENCES 





Post-Test 

Experimental 

Group 

Mean N SD 


Post-Test 

Control 

Group 

Mean N SD 


Difference of 
Experimental 
Minus Control 
Group Means 


Z-Score of 
Difference 
(1.65 needed 
For .05 Level 
Of Signifi- 
cance) 


All 

Colleges 


31.8 


92 


6.59 


30.4 


112 


6.25 


+1.4 


+1.54 


Cypress 

College 


28.7 


27 


6.06 


29,2 


44 


6.47 


-0.5 


-0.34 


Mount San 

Antonio 

College 


29.9 


22 


5.56 


29.6 


38 


5.66 


+0.3 


+0.20 


Orange 

Coast 

College 


34.7 


43 


6.26 


33.1 


30 


5.84 


+1.6 


+1.10 
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treatment, but would caution against large resource allocation for such purposes 
until further research could substantiate test results. 

Tables II and III tabulate data used to test as gains in scores from 

pre-test to post-test averages are compared. While one might well expect any 

t 

student in any economics course to improve in economic understanding over a 
semester period, the objective here was to compare relative rates of gain 
between control and experimental groups. On this basis H02 was soundly accepted 
(A Z-seore of -.13.), and no significant effect is evidenced by Table II. Table 
HI gives the same "mo significant effect" finding though in a slightly different 
manner, in that Z-score differences themselves were compared. While there were 
obvious gains in economic understanding (Z-scores ranged from +2.24 to +8.16, 
though only +1.65 was necessary for a .05 level of significance.), experimental 
groups show no significant gains over control groups. 

The lack of significant effect from instructional objectives on Hog is 
somewhat mitigated, however, by inspection of Table IV. For an examination of 
pre-test averages revealed a most "unplanned" research design impurity: Exper- 

imental groups had significantly higher pre-test means than control groups. 

Of course, the implication of a higher pre-test mean on H02 was that the amount 
of gain from pre-test to po3t-test would tend to be reduced, and thus lower 
significant effects on testing H02 (It does appear rather suprising that Ho^ 
would have a .06 significance level, while Ho 2 had only a .13?). In this same 
vein, another test "impurity" (Though inherent, as instructors determine their 
own class conduct*) arose because participating economists weighted post-test 
results differently in computing semester grade results. As might be expected, 
the college scoring highest on the post-test was the one in which the instructor 
considered the post-test score as the final exam grade, while the college 
scoring lowest was that in which the economist used the post-test simply as a . 
"warm-up" to finals with no grade credit. The age-old conclusion is that 
grades do produce significant results? 



TABLE II 



COMPARISON OF PRE-TEST TO POST-TEST 
GAIN DIFFERENCES 





Pre-Test to 
Post-Test Gain, 
Experimental 
Group 

Mean N SD 


Pre-Test to 
Post-Test Gain, 
Control 
Group 

Mean N SD 


Difference of 
Experimental 
Minus Control 
Group Means 


Z-Score of 
Difference 
(1.65 needed 
For .05 Level 
of Signifi- 
cance ) 


All 

Colleges 


+5.3 


92 


5.54 


+5 . 4 


112 


5.43 


-0.1 


-.13 


Cypress 


















College 


+3.7 


27 


5.18 


+4.3 


44 


5.36 


-0.6 


-.48 


Mount San 

Antonio 

College 


+3.8 


22 


4.52 


+5.4 


38 


5.01 


-1.6 


-1.27 


Orange 

Coast 

College 


+7.1 


43 


5.82 


+7.0 


30 


5.54 


+0.1 


+.07 
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TABLE III 



COMPARISON OF PRE-TEST TO POST-TEST 
GAIN DIFFERENCES 





Pre-Test 

Score 

Mean N SD 


Post-Test 

Score 

Mean N SD 


Difference 
of Post-Test 
Minus Pre- 
Test Scores 


Z-Score of 
Difference 
(1.65 needed 
For . 05 Level 
of Signifi- 
cance) 


All Colleges: 
Experiment 
All enrolled 


25.5 


135 


6.27 


31.8 


92 


6.59 


+6.3 


+7.16 


Completed 


26.5 


92 


5.91 


31.8 


92 


6.59 


+5.3 


+5.70 


Control 
All enrolled 


24.2 


172 


6.25 


30.4 


112 


6.25 


+6.2 


+8.16 


Completed 


25.0 


112 


6.16 


30.4 


112 


6.25 


+5.4 


+6.51 


Cypress 
College: 
Experiment 
All enrolled 


23.6 


43 


6.31 


28.7 


27 


6.06 


+5.1 


+3.45 


Completed 


25.1 


27 


6.02 


28.7 


27 


6.06 


+3.6 


+2.29 


Control 
All enrolled 


23.9 


84 


6.24 


29.2 


44 


6.47 


+5.3 


+4.42 


Completed 


24.9 


44 


5.91 


29.2 


44 


6.47 


+4.3 


+3.21 


Mount San 
Antonio 
College 
Experiment 
Ail enrolled 


25.6 


33 


5.13 


29.9 


22 


5.56 


+4.3 


+2.83 


Completed 


26.1 


22 


5.49 


29.9 


22 


5.56 


+3.8 


+2.24 


Control 
All enrolled 


23.7 


55. 


5.91 


29.6 


38 


5.66 


+5.9 


+ 4.96 


Completed 


24.2 


38 


5.78 1 


29.6 


38 


5.66 


+5.4 


+4.29 


Orange Coast 
College: 
Experiment 
All enrolled 


26.9 


59 


6.44 


34.7 


43 


6 . 26 


+7.8 


+6.00 


Completed 


27.7 


43 


5.86 


34.7 


43 


6 . 26 


+7.0 


+5.22 


Control 
All enrolled 


25.8 


33 


6.54 


33.1 


30 


5.84 


+7.3 


+4.59 


Completed 


26.2 ' 


30 


6.66 


33.1 


30 


5.84 


+6.9 


+4.18 



TABLE IV 



COMPARISON OF PRE-TESTS FOR 
GROUP DIFFERENCES 





Pre-Test, 

Experimental 

Group 

Mean N SD 


Pre-Test 

Control 

Group 

Mean N SD 


Difference or 
Experimental 
Minus Control 
Group Pre- 
Test 


Z-Score of 
Difference 
(1.65 needed 
For . 05 Level 
of Signifi- 
cance) 


All 

Colleges : 

All enrolled 


25.5 


135 


6.27 


24.2 


172 


6.25 


+1 . 3 


+1.81 


Completed 


26.5 


92 


5.91 


25.0 


112 


6.16 


+1.5 


+1.76 


Cypress 
College: 
All enrolled 


23.6 


43 


6.31 


23.9 


84 


6.24 


-0.3 


-0.25 


Completed 


25.1 


27 


6.02 


24.9 


44 


5.91 


+0.2 


+0.14 


Mount San 
Antonio 
College: 

All enrolled 


25.6 


33 


5.13 


23.7 


55 


5.91 


+1.9 


+1.57 


Completed 


26.1 


22 


5.49 


24.2 


38 


5.78 


+1.9 


+1.27 


Orange 

Coast 

College: 

All enrolled 


26.9 


59 


6 . 44 


25.8 


33 


6.54 


+i.i 


+0.76 


Completed 


27.7 


43 


5.86 


26.2 


30 


6.66 


+1.5 


+0.97 
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Tables V and VI list additional research generated as side effects in 
testing Ho-^ and Ho 2 , and compare withdrawal rates, age, and entrance test scores 
(as made available by participating colleges). It should be no^ed that an 
initial assumption was made of random student enrollment (An assumption dubious 
in light of Table IV. ) , and test results were assumed to be the effect of 
instructional objectives only. While some of the data in Tables V and VI is 
inconclusive (ACT/SCAT scores were not available for many students.) at best, 
it did not appear that such student characteristics would have affected test 
results to any significant degree. 

Summary 

In light of the introductory statements made regarding the deficiency of 
research on instructional objectives effect, perhaps a modest claim may be made 
for some contribution by this research study. While the specific null^ 
hypotheses set were both accepted at the .05 level of significance (And, there- 
fore, a significant effect claim could not be made.), a .06 level of significance 
was achieved on Ho 1 , and test "impurities" may have impaired the test on Ho 2 . 
Given the educational literature theorizing the learning gains from use of 
instructional objectives, and reviewing the test findings of this research 
study, it would appear that further research on instructional objectives effect 
is both necessary and desirable. 

Dr. James A. Phillips 
Cypress College 
Cypress, California 




10 

10 



TABLE V 



COMPARISON OF PRE-TESTS FOR 
WITHDRAWAL EFFECTS 





Pre-Test, 

All enrollec 

Mean N SD 


Pre-Test, 

Completed 

, t 

Mean N SD 


Difference of 
Completed 
Minus All 
enrolled Pre- 
Test 


Z-Score 
Difference 
(1.65 needed 
For .05 Level 
of Signifi- 
cance) 


All 

Colleges : 
Experiment 


25.5 


135 


6.27 


26.5 


92 


5.91 


+1.0 


+1.22 


Control 


24.2 


172 


6.25 


25.0 


112 


6.16 


+0.8 


+1.07 


Cypress 

College: 

Experiment 


23.6 


43 


6.31 


25.1 


27 


6.02 


+1.5 


+1.02 


Control 


23.0 


84 


6.24 


24.9 


44 


5.91 


+1.0 


+0.88 1 


Mount San 
Antonio 
College: 
Experiment 


25.6 


33 


5.13 


26.1 


22 


5.49 


+0.5 


+0.33 


Control 


23.7 


55 


5.91 


24.2 


38 


5.78 


+0.5 


+0.42 


Orange 

Coast 

College: 

Experiment 


26.9 


59 


6.44 


27.7 


43 


5.86 


+0.8 


+0.65 


Control 


25.8 


33 |6.54 


26.2 


30 


6 . 66 


+0.4 


+0.24 
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TABLE VI 



STUDENT CHARACTERISTICS 





Age 


Withdrawal 




ACT or 
















SCAT Scores 




Mean 


N 


Start 


End 


% Drop 


N 


Score 




(yrs^ 








(%) 






Cypress 

College: 












14 


29.3 - SCAT (%ile) 


Experiment 


19.4 


43 


43 


27 


37.2 


20 ‘ 


'17.7 - ACT (%ile) 














24 


33.0 - SCAT (%ile) 


Control 


19.5 


84 


84 


44 


47.6 


51 


17.3 - ACT (%ile) 


Mount San 
















Antonio 

College: 












24 


288.5 - SCAT (Total) 


Experiment 


19.9 


33 


33 


22 


33.3 


3 


19.3 - ACT (%ile) 














30 


298.3 - SCAT (Total) 


Control 


20.2 


55 


55 


38 


30.9 


4 


19.8 - ACT (%ile) 


Orange Coast 
College: 














(Total 


Experiment 


21.6 


59 


59 


43 


27.1 


50 


53.6 ACT %ile) 


Control 


20.5 


33 


33 


30 


9.1 


— 


— 


All Colleges: 
Experiment 


20.5 


135 


135 


92 


31.9 




— — — 


Control 


19.9 


172 


172 


112 


34.9 




— 


Combined 


20.2 


307 


307 


204 


33.6 


— 


— 
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